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1  Status  of  Effort 

This  is  an  AASERT  grant  that  is  a  supplementary  award  to  the  parent  AFOSR  Grant  FR9620- 
94-1-002.  The  objective  of  the  AASERT  grant  was  to  train  a  graduate  student  who  must  be 
a  US  citizen.  We  were  highly  successful  in  our  objectives  -  the  graduate  student  Douglas  E. 
Heisterkamp  successfully  completed  all  the  requirements  for  a  Ph.D.  and  also  published  several 
journal  papers  (see  the  list  at  the  end  of  the  report).  Also,  he  gave  oral  presentations  in  two 
international  conferences,  one  in  Minneapolis,  and  the  other  one  in  the  Vienna,  Austria.  The  PI 
made  a  considerable  effort  in  training  the  AASERT  graduate  student  in  the  area  of  the  project. 

The  accomplishments  given  below  overlap  with  some  of  the  accomplishments  of  the  parent 
AFOSR  grant  of  which  the  present  grant  was  a  supplementary. 


2  Accomplishments 

We  have  constructed  mutual  invariants  of  families  of  coplanar  conics.  These  invariants  are  compared 
with  the  use  of  invariants  of  two  conics  and  a  case  is  presented  where  the  proposed  invariants  have 
a  greater  discriminating  power  than  the  previously  used  invariants.  The  use  of  invariants  for  two 
conics  is  extended  to  any  number  of  coplanar  conics.  A  lambda-matrix  is  associated  with  each  family 
of  coplanar  conics.  The  use  of  lambda-matrices  is  extended  from  the  single  variable  polynomial  to 
multi-variable  polynomials.  The  Segre  characteristic  and  other  invariants  of  the  lambda-matrix  are 
used  as  invariants  of  the  family  of  conics. 

Conics  are  widely  recognized  in  the  study  of  machine  vision  as  the  most  fundamental  image 
features  next  to  lines.  Many  natural  and  man-made  objects  have  circular  shapes,  and  in  addition, 
many  other  curves  can  be  approximated  by  conics.  Arbitrary  planar  shapes  can  be  represented 
by  a  set  of  coplanar  conics.  There  has  been  much  work  done  in  recognizing  pairs  of  coplanar 
conics  using  invariants  Forsyth  et  al  (1991),  Maybank  (1993).  The  invariants  proposed  by  us  can 
discriminate  between  two  non-projectively  related  families  of  coplanar  conics  in  which  the  previously 
used  invariants  give  the  same  value. 

Invariants  are  properties  of  geometric  configurations  which  remain  unchanged  under  an  appro¬ 
priate  class  of  transformations.  In  fact,  it  has  been  asserted  that  invariance  is  the  essential  property 
of  a  shape  description.  Mutual  invariants  are  properties  of  a  set  of  objects  that  remain  unchanged 
under  a  class  of  transformations.  Cooper  et.  al.  (1994)  used  mutual  invariants  in  recognition  of 
objects  in  aerial  photographs.  In  a  similar  vein,  the  mutual  invariants  of  a  family  of  coplanar  conics 
can  be  used  in  recognizing  objects  by  being  shape  descriptors  to  index  a  model  of  the  object  from 
a  database.  Previous  work  with  invariants  from  conics  have  focused  exclusively  on  two  coplanar 
conics.  Often,  an  object  will  have  more  than  two  coplanar  conics.  This  then  leads  to  the  question 
of  using  mutual  invariants  of  more  than  two  coplanar  conics.  In  this  paper  we  address  that  question 
by  extending  the  use  of  invariants  to  recognizing  objects  with  any  number  of  coplanar  conics. 

The  invariants  of  coplanar  conics  may  be  used  as  a  descriptor  for  a  planar-surface  of  an  object. 
An  object  can,  in  general,  have  multiple  planes  with  coplanar  conics  from  which  invariants  can  be 
obtained.  The  set  of  these  invariants  would  comprise  a  description  of  the  object.  It  is  desirable  to 
have  as  many  invariants  of  an  object  as  possible  to  index  to  the  model,  since  not  all  or  even  any 
conics  of  a  particular  plane  may  be  visible  in  a  given  view.  To  handle  the  case  when  some  of  the 
conics  of  a  family  are  occluded,  only  a  little  more  work  is  needed  to  calculate  the  invariants  of  a 
subsets  of  the  family  of  conics.  These  invariants  can  then  be  used  to  give  a  list  of  candidate  objects 
of  which  the  visible  part  may  be  a  subcomponent. 
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The  invariants  we  are  interested  in  are  the  projective  invariants.  A  projective  invariant  is  an 
attribute  of  an  object  that  will  remain  the  same  under  changes  in  pose  and  camera  calibrations 
since  perspective  transformations  are  contained  in  the  set  of  projective  transformations. 

Astrom  (1994)  has  shown  some  of  the  limitations  of  planar  projective  invariants.  Although 
valid  limitations,  we  take  the  position  that  they  do  not  affect  the  practical  application  of  planar 
invariants.  First,  due  to  the  extreme  nature  of  the  series  of  projections  needed  to  cause  the  failure 
of  the  projective  planar  invariants,  they  should  be  very  rare  in  most  uses.  Even  though  each 
projection  in  the  series  can  be  realized  by  the  pinhole  camera  model,  the  composition  of  the  series 
of  projection  will,  in  general,  not  be  realizable  by  a  single  imaging  of  a  pinhole  camera.  This  means 
for  the  failure  to  occur,  the  application  would  be  of  an  image  which  contains  a  series  of  images, 
each  one  of  which  is  contained  in  the  previous  image.  This  occurrence  will  be  rare  or  nonexistent  in 
many  applications.  Second,  invariants  have  been  used  to  guide  the  search  for  a  match  in  a  object 
database.  Most  of  the  time,  a  set  of  possible  matches  will  be  found  and  an  additional  verification 
step  is  needed  to  chose  from  the  possible  matches.  The  verification  step  should  catch  the  rare 
mismatches  due  to  the  limitations  presented  by  Astrom. 

A  matrix  with  polynomial  entries  is  called  a  lambda-matrix.  The  lambda-matrix  has  been  used 
in  physics  and  system  control  theory  but  we  believe  that  this  is  the  first  time  we  are  using  them  in 
the  field  of  computer  vision.  We  use  a  lambda-matrix  as  a  tool  for  calculating  the  invariants  of  a 
family  of  coplanar  conics.  The  polynomials  will  be  in  the  indeterminates  Ai,  A2, . . . ,  An  where  n  will 
depend  on  the  problem  at  hand,  and  the  coefficients  of  the  polynomials  will  lie  in  the  field  of  complex 
numbers.  We  construct  3x3  symmetric  lambda-matrices  where  the  number  of  indeterminates  is 
the  number  of  conics  in  the  family  of  coplanar  conics.  When  needed,  we  make  a  restriction  to  using 
only  3x3  symmetric  with  polynomials  of  degree  at  most  1.  In  moving  to  using  multi- variable 
polynomials  from  the  single  variable  polynomials,  we  are  moving  from  an  Euclidean  Domain  to  an 
Unique  Factorization  Domain. 

The  noise  in  an  image  may  affect  the  coefficients  of  the  conics  that  are  found  in  the  image.  How 
the  noise  affects  the  coefficients  of  a  conic  is  dependent  on  the  method  used  to  find  and  fit  the  conic. 
To  isolate  the  effects  of  noise  on  the  invariants,  we  add  simulated  noise  to  ellipses  and  compare  the 
invariants  of  the  noisy  ellipse  and  the  original  ellipse.  Instead  of  adding  noise  to  the  coefficients 
of  the  conic,  we  added  an  percentage  of  error  to  the  lengths  of  the  major  and  minor  axis  and  to 
the  angle  of  rotation.  This  resulted  in  a  realistic  error  in  the  fitting  of  a  conic.  A  series  of  ellipses 
were  randomly  generated  and  the  true  invariants  were  calculated.  Noise  was  then  added  to  each 
axis  by  randomly  varying  the  axis  length  in  the  range  of  ±  error  percentage.  The  angle  of  rotation 
was  randomly  varied.  The  invariants  of  the  noisy  ellipse  were  calculated  and  the  absolute  percent 
difference  was  recorded.  The  process  was  conducted  for  500  ellipses  and  the  average  percentage 
of  difference  between  the  noisy  ellipses  and  the  original  ellipse  were  calculated.  It  is  interesting  to 
note  that  not  all  invariants  are  created  equal  in  their  ability  to  handle  noise. 

2.1  Applications  of  Legendre  Transformation  to  Computer  Vision 

The  Legendre  transformation  is  a  well  known  concept  in  classical  theoretical  physics.  Recently, 
I.  Weiss  of  the  Center  for  Automation  Research,  University  of  Maryland,  has  applied  it  to  study 
problems  of  invariance  in  computer  vision;  his  results  were  presented  in  the  1994  and  1995  ARPA 
Image  Understanding  Workshops.  Most  authors  treat  the  Legendre  transformation  (LT)  purely 
analytically  .  Only  the  famous  Russian  theoretical  physicist  Arnold  (1978)  has  treated  the  LT 
from  a  geometric  point  of  view;  also  a  very  general  version  of  the  LT  for  a  hypersurface  and  the 
relationship  with  symplectic  invariants  are  discussed  a  monograph  by  Hofer  and  Zehnder  (1994). 


3 


In  this  paper  we  obtain  some  new  geometric  properties  pf  the  LT  that  are  of  potential  interest  in 
image  understanding.  We  show  that  under  the  LT  the  cross  ratio,  and  the  connectedness  property 
remain  invariant.  Also,  we  show  that  the  LT  maps  a  conic  into  a  conic.  As  an  application  we 
develop  an  algorithm  to  detect  corners  in  images.  We  also  discuss  the  relationship  of  the  LT  with 
the  classical  Hough  transform. 

2.2  Matching  of  3-D  Polygonal  Arcs 

Matching  of  arcs  is  a  basic  problem  in  computer  vision.  It  is  used  in  many  applications,  e.g.,  indus¬ 
trial  parts  inspection,  character  recognition,  analysis  of  engineering  drawings,  etc.  The  problem  of 
finding  an  approximate  match  between  short  arcs  and  pieces  of  a  long  arc  is  known  as  the  segment 
matching  problem.  There  have  been  several  approaches  to  this  problem.  These  can  be  classified  as 
global  or  local  in  nature.  In  the  global  approaches,  a  set  of  global  features  are  obtained  for  each  arc 
and  we  match  these  features  for  the  two  arcs.  The  3-D  generalization  of  the  matching  of  polygonal 
arcs  is  of  considerable  interest  in  computer  vision:  the  problem  has  applications  in  industrial  parts 
inspection,  motion  estimation,  structure  determination  of  macromolecules  from  electron  density 
maps,  and  in  many  other  3-D  vision  problems.  Also,  a  polygonal  approximation  provides  a  data 
compression  of  images  and  several  contour  segmentation  techniques  have  been  proposed  which  have 
applications  in  the  design  of  the  vision  system  of  a  mobile  robot. 

There  are  several  novel  features  in  our  approach  that  distinguishes  it  from  previous  works  on 
matching  of  3-D  curves.  Unlike  the  previous  works,  we  use  here  unit  quaternions  to  denote  3-D 
rotations  that,  as  is  well  known,  has  the  advantage  (over  other  representations  for  rotations)  in 
giving  a  closed  form  solution.  Further,  we  proved  a  new  result  interpreting  the  extreme  values  of 
the  distance  measure  for  two  3-D  polygonal  axes  of  equal  lengths  as  the  eigenvalues  of  a  certain 
(well  defined)  4x4  positive  semidefinite  matrix;  the  minimum  value  of  the  distance  measure 
corresponding  to  the  minimum  eigenvalue  of  this  matrix.  It  follows  that  the  matching  problem  of 
3-D  curves  can  now  be  reduced  to  the  purely  algebraic  problem  of  studying  the  eigenvalues  of  a 
certain  matrix.  We  applied  the  same  approach  to  matching  2-D  polygonal  arcs.  We  showed  that  a 
subgroup  of  the  unit  quaternions  can  be  used  to  denote  2-D  rotations.  Using  this  subgroup  of  the 
unit  quaternions,  we  proved  that  the  minimum  value  of  the  the  distance  measure  between  two  2-D 
polygonal  arcs  is  the  smallest  eigenvalue  of  the  2x2  submatrix  located  in  the  lower  right  quadrant 
of  the  4x4  matrix  used  in  3-D  matching.  It  follows  that  the  matching  problem  of  2-D  curves 
can  now  be  reduced  studying  the  eigenvalues  of  matrix.  We  then  could  apply  standard  methods  of 
numerical  linear  algebra  to  estimate  the  eigenvalues  and  hence  the  matchings  (up  to  any  desired 
degree  of  accuracy).  Our  algorithm  is  practical  to  implement  and  we  have  carried  out  extensive 
implementations. 


3  Personnel  Supported 

Doug  Heisterkamp,  Ph.D.  student,  12  months.  (As  per  the  conditions  of  an  AASERT  grant,  the 
student  is  a  US  citizen.) 

4  Interactions/Transitions 

The  results  from  our  effort  could  be  used  in  developing  technology  for  automatic  target  recognition. 
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5  Presentations  in  Conferences 


1.  April,  1996,  IEEE  International  Conference  on  Robotics  and  Automation,  Minneapolis,  Min¬ 
nesota.  Oral  presentation. 

2.  August,  1996,  International  CAonference  on  Pattern  Recognition  (ICPR),  Vienna,  Austria. 
Oral  presentation. 

5.1  Interaction  with  Air  Force  Laboratories 

The  PI  gave  a  presentation  of  his  work  at  the  Wright-Patterson  Air  Force  Laboratory,  Dayton,  OH 
in  1995.  He  gave  two  talks  on  successive  days  and  also  held  detailed  discussions  with  some  staff 
members  including  V.  Velten  and  J.  Leonard. 

The  PI  was  invited  to  participate  in  the  Smart  Sensors  Workshop  held  in  1996  at  the  Air 
Force  Laboratory  at  the  Kirtland  Air  Force  Base  in  Albuquerque,  NM.  He  also  gave  a  poster 
presentation  at  the  workshop.  During  the  workshop,  he  benefited  greatly  by  interacting  with  the 
other  participants  including  several  Air  Force  staff. 


6  Technical  Publications 

6.1  Journal  Papers 

1.  D.  Heisterkamp  and  P.  Bhattacharya,  “Matching  of  Three-Dimensional  Polygonal  Arcs,” 
IEEE  Trans.  Pattern  Analysis  and  Machine  Intelligence,  vol.  19,  no.  1,  pp.  68-73,  (1997). 

2.  D.  Heisterkamp  and  P.  Bhattacharya,  “Invariants  of  Families  of  Coplanar  Conics,  and  their 
Applications  to  Object  Recognition,”  Journal  of  Mathematical  Imaging  and  Vision,  vol.  7, 
pp.  253-267,  (1997). 

3.  D.  Heisterkamp  and  P.  Bhattacharya,  “Matching  of  2-D  Polygonal  Arcs  Using  a  Subgroup  of 
the  Unit  Quaternions,  Computer  Vision  and  Image  Understanding ,”  accepted  for  publication. 

4.  D.  Heisterkamp  and  P.  Bhattacharya,  “Geometric  Properties  of  Legendre  Transformations,” 
Computer  Vision  and  Image  Understanding,  tentatively  accepted  for  publication,  subject  to 
revision. 


6.2  Refereed  Conference  Papers 

5.  D.  Heisterkamp  and  P.  Bhattacharya,  “Matching  of  3-D  Curves” ,  Proc.  IEEE  Internat.  Conf. 
Robotics  and  Automation,  Minneapolis,  April,  1996,  pp.  3490-3495. 

6.  D.  Heisterkamp  and  P.  Bhattacharya,  “Invariants  of  a  Family  of  Coplanar  Conics  and  Object 
Recognition,”  Proc.  13th  Internat.  Conf.  Pattern  Recognition  (ICPR’96),  Vienna,  Austria, 
August,  1996,  Computer  Vision  Track,  pp.  677-681. 


7  Patent  disclosures 

None. 
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8  Honors/ A  wards 

’  9 

1.  National  Lecturer  of  the  Association  of  Computing  Machinery  (ACM)  during  1996-99. 

2.  Distinguished  Visitor  of  the  IEEE  Computer  Society  during  1996-99. 

3.  Award  for  highest  involvement  from  the  Nebraska  chapter  of  the  IEEE  Computer  Society, 
1996. 

4.  Certificate  for  being  an  outstanding  member  from  the  Nebraska  chapter  of  the  IEEE  Com¬ 
puter  Society,  1996. 

5.  Editorial  board  member  of  the  IEEE  Computer  Society  Press  during  1996-98. 

6.  Editorial  board  member  of  “Pattern  Recognition”  during  1994-present. 

7.  Elected  the  Chairman  of  the  IEEE  Computer  Society,  Nebraska  Chapter  for  1995-96,  and 
re-elected  for  1996-97. 
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